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^P ' Abstract. Let _F be a genus g curve and a : F ^ F a, real structure 

with the maximal possible number of fixed circles. We study the real 
moduli space TV' = Fix (a*) where a* : Af ^ Af is the induced real 
structure on the moduli space TV of stable holomorphic bundles of rank 
2 over F with fixed non-trivial determinant. In particular, we calculate 
H*{J\f',Z) in the case of g = 2, generalizing Thaddeus' approach to 
computing iif*(TV, Z). 
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Cg . 1. Introduction 

In their influential paper [1], Atiyah and Bott used two-dimensional 
Yang-Mills theory to compute cohomology of the moduli space Af of stable 

r-^ ■ holomorhic bundles over a Riemann surface F. In essence, the computation 

^^ ' was inspired by the idea that the Yang-Mills functional in this dimension 

should be an equivariantly perfect Morse-Bott function (this was proved 

^ I later in full generality by Daskalopoulos [2]). 

00 ! A real structure a : F ^ F induces a real structure cr* on the mod- 



o 



X 



uli space M. Understanding the structure of the real moduli space A/"' = 
Fix (cj'^) is an important but subtle problem. It has been addressed most 
recently in a series of papers by N.-K. Ho and C.-C. M. Li, the latest be- 
ing [6j. They mainly treat the case when the real structure a has no fixed 
points, which leads them to the study of the Yang-Mills functional on the 
non-orientable surface F/a. 

In this paper we consider the other extreme, when a has the maximal 
possible number of fixed circles. In this case, the pair (F, a) is usually 
referred to as an M~curve. More specifically, we work with the moduli space 
M of stable holomorphic bundles of rank 2 over F with fixed non-trivial 
determinant, and with the associated real moduli space A/"'. Instead of the 
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infinite dimensional Yang-Mills functional we utilize a finite dimensional 
Morse-Bott function as in Thaddeus [S]. The perfection of the function 
in Thaddeus' paper was suggested by the work of Frankel [1| while the 
perfection of ours is suggested by Duistermaat's paper [3]. 

For technical reasons, in this paper we will only take up the case of genus 
two M-curves, hoping to address the case of higher genera elsewhere. For 
M-curves of genus two, the second author [9j described TV' algebraically as 
the intersection of two quadrics in the five-dimensional real projective space; 
however, it proved to be rather difficult to extract any further information 
about the topology of TV' from that description. The Morse theoretic ap- 
proach of this paper, on the other hand, gives a complete calculation of the 
integral cohomology of TV'. 

Theorem 1.1. Let {F,a) be a genus two M-curve and Af' the real moduli 
space of stable holomorphic bundles of rank 2 over F with fixed non-trivial 
determinant. Then, at the level of graded abelian groups, there is an iso- 
morphism H*{J\f',Z) = H*{S^ X S^ X S\Z). 

It should be mentioned that the above isomorphism does not hold at the 
level of cohomology rings, see Remark llO.il 

Parts of this project were accomplished while we attended the Summer 
2006 Session of the Park City/IAS Mathematics Institute. We express our 
appreciation to the organizers for providing a stimulating environment. 

2. Real moduli spaces over curves 

Let F be a compact surface of genus g > 2 and fix a point p ^ F. Denote 
by TV the moduli space of stable holomorphic bundles £ ^ F of rank 2 with 
determinant L" , where p & F is viewed as a divisor. Then TV is a smooth 
complex manifold of real dimension 6(7 — 6, modeled on the complex vector 
space H^{F, Ends) by the deformation theory. 

Let (T : -F — > -F be a real structure on F whose real part F' = Fix (a) is 
non-empty. Then F' contains at least one circle, but may contain as many 
as g + 1 circles. In the latter case, the pair (F, a) is called an iVf -curve. 
Choose p £ F' then a : F —>■ F induces an involution a* : TV ^ TV by the 
formula 

a*[£] = [a*S], 



where 8 stands for the complex conjugate of 8. Note that since a : F ^>- F 
is orientation reversing, a*hp = hp, thus making the complex conjugation 
in the above formula necessary. 

Note that the linear map H^{F,End8) -^ H^ {F,End{a*8)) induced by 
o" is a complex conjugation, therefore, the involution cj* : AA — > A/" is anti- 
holomorhic and hence is a real structure on Af. It follows that the real moduli 
space J\f' = Fix {cr'^) is a closed smooth manifold of dimension 3g — 3. 

Proposition 2.1. The manifold Af' is orientable. 

Proof. According to [T] , the moduli space Af is simply connected and spin. 
Therefore, the real structure o"'^ must be compatible with the unique spin 
structure on TV in the sense of [TO] . The main result in [10] then applies to 
infer that Af' is orientable. Note that a* need not preserve the spin struc- 
ture in the usual sense, since it is orientation reversing when the complex 
dimension of Af is odd, that is, when g is even. D 



3. Representation varieties 

Let Fq be the surface F punctured at p € -F. The theorem of Narasimhan 
and Seshadri [7J can be used to identify Af with the representation variety AA 
which consists of the conjugacy classes of SU{2) representations of 7ri(Fo) = 
7ri(Fo,xo) sending [9Fo] G vri(Fo) to —1 G SU{2) (the latter condition does 
not dependent on the choice of basepoint because —1 is a central element in 
SU{2)). 

Given a real structure a : F ^ F with non-empty F' = Fix (a) and 
p ^ F' as above, choose a basepoint xq € F' r\ Fq. Then we have an 
induced involution a^, : 7ri(i<o) -^ 7ri(Fo), which in turn induces an involution 
a* : AA ^ A4hy the formula 

(J* \a\ = \ao aA. 



That a* is well defined follows from the fact that a^, [OFq ] = [BFq ] -"^ = — 1. 
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Lemma 3.1. Let ip : Ai —^ Af be the Narasimhan-Seshadri diff'eomorphism 
then the following diagram commutes 
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Proof. The Narasimhan-Seshadri correspondence assigns to every [a] € A^ a 
stable holomorphis bundle iSq ^ F as follows. Let Fq -^ Fq be the universal 
covering space of Fq and consider the holomorphic bundle 8 ^ Fq with 

£ = Fq x^^(^g) C , 

where 7ri(Fo) acts on C^ via a : vri(Fo) -^ SU{2). Obviously, det<S is trivial. 
Since the holononiy of a along the loop OFq is fixed, we can trivialize £ 
near the boundary OFq. Glue D^ x C^ in using the transition function z~^ 
along OFq. The result is the stable bundle £a ^ F with the determinant 
det £a = L~ yielding the Narasimhan-Seshadri correspondence. 

For any given [a] G A4 we have a* [£a] = [a* £a] = [£a*a], where a is 
the complex conjugate of a. However, for any matrix 
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G SU{2), 



its complex conjugate 
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is the matrix conjugate of A via 



J 



1 

-1 



G SUi2) 



(we use the standard identification between SU{2) matrices and unit quater- 
nions). This means that a and a are conjugate representations, and there- 
fore the bundles £a*a and £a*a are isomorphic. D 

Denote by M' the fixed point set of the involution a* : M ^t AA then the 
above lemma implies that M' and M' are diffeomorphic. 



Corollary 3.2. The moduli space M! is a smooth closed orientable m,anifold 
of dim,ension 3g — 3. 

In conclusion, note that A^ is a symplectic manifold with symplectic form 
Lo: H^{F;d.dp) H^{F;adp) ^ M given hj u{u,v) = -1/2 tr (n U v)[F], 
see Goldman [5j. 

Lemma 3.3. The m,ap a* : A4 ^ M is a real sructure with respect to the 
symplectic form lo, that is, a*LU = —to. 

Proof. This result is obtained by the following straightforward calculation : 

{a*uj){u, v) = uj{a*u, a*v) = -1/2 tr {a*u U a*v) [F] 

= -1/2 tr(o-*(n U v))[F] = -1/2 tr (n U v) a^[F] = -uj{u,v). 

In the last equality, we used the fact that the map a : F ^ F is orientation 
reversing. D 

4. The case of 51 = 2 

Let {F, a) be a genus two M-curve embedded in M^ as shown in Figure 
[H with a : F ^ F acting as the reflection in the horizontal plane fixing the 
obvious three circles. Let Fq be a once punctured surface F, the puncture 
p £ F being a real point on the left circle of F' (the other two options for 
positioning p will be treated in Section [TO]) . Then 7ri(Fo) is the free group 

TTi{Fo) = {ai,bi,a2,b2\ ) 

whose generators ai, 02, bi, &2 are shown in the picture. Also shown is the 
curve c = [ai,6i] • [a2i&2]- All of the above curves are oriented clockwise 
with respect to the projection shown in Figure [H 

The moduli space A4 can now be described as follows. Let us consider the 
smooth map /i : SU{2)^ -^ SU{2) given by /x(yli,Bi, A2, S2) = -[Ai,Bi] ■ 
[^42,-62]. Since 1 E SU{2) is a regular value of //, its preimage /i~^(l) is a 
smooth manifold. It contains no reducibles, that is, 4~tuples {Ai,Bi, A2, B2) 
whose entries commute with each other, because the latter would contradict 
the equation [^i,i?i] • [^2,-62] = —1- Therefore, the action of SO{3) = 
SU{2)/±1 on /i^^(l) by conjugation is free, and its quotient space is then the 
smooth six-dimensional manifold A4. A straightforward calculation shows 



p 




Figure 1. 

that the induced action a : 7ri(i<o) — > 7ri(i<o) is given on the generators by 
the formulas 

a^:{ai) = ai, a^{bi) = b^^, 

cr*{a2) =b^^cb2a2b2^bi, cr*(62) = b]^^b2^bi. 

In practical terms, A^ consists of the conjugacy classes [Ai,Bi,A2,B2] of 
quadruples (Ai,Bi,A2,B2) such that [yli,i?i] • [A2,B2] = —1, and the real 
structure a* : A4 ^ Ai is given by 

a*[Ai,Bi,A2,B2] = [BiAiB^\ B^\ -52^2^2^', B^^ 

As we already know, the fixed point set M' of o"* : A^ ^ A^ is a smooth 
orientable manifold of dimension 3. 

5. The function 

Let / : TW ^ E be the function on the moduli space M defined by the 
formula 

f{[Ai,Bi,A2,B2])=iT{Bi)/2. 

This is a smooth function whose range is [—1,1], and /~^(— 1,1) is acted 
upon by S^ as follows. After conjugation if necessary, every element of 
/^-'^(— 1,1) can be written in the form [Ai, Si, 7^2, -B2] with Bi = e*^ and 
< /3 < vr. The circle action is then given by e^^ : [Ai,Bi,A2-,B2\ 1-^ 
[Aie'^,BuA2,B2]. 

Lemma 5.1. This is a well defined action on /~^(— 1,1). 

Proof. Any other choice of representative in [Ai,Bi,A2,B2] with Bi = e*'*' 
and < 7 < vr, will have the property that e*''' = ge^^g^^ for some g G 
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SU{2). Therefore, 7 = /3 and 5 is a unit complex number. Because of that, 
{gAig~^)e^^ = g{Aie'^^)g~^, making the action weh defined. Of course, if 
Bi = ±1, the conjugating element g can be any SU{2) matrix, and the 
above argument fails. D 

According to Goldman [5], the above circle action preserves the symplectic 
form u; on A1 and arccos / is its moment map (up to a factor of i). 

It is clear that f o a* = f hence / : A^ ^ M is invariant with respect to 
a* : M ^ M. The restriction of / to M' will be denoted by /' : M' -^ M. 
The range of /' is again [—1,1]. The circle action on /~^(— 1,1) is not 
defined on (/')^^(— 1, 1). Nevertheless, it gives rise to the residual Z/2 
action r : A4' -^ A4' defined on the entire moduli space A4' (and in fact on 
the full moduli space A4) by the formula 

r{[AuBi,A2,B2]) = [-Ai,BuA2,B2]. (1) 

6. The critical submanifolds of /' 

Thaddeus [8] proved that / : A^ — > M is a Morse-Bott function and 
described its critical submanifolds. These critical submanifolds are acted 
upon hy a* : Ai ^ Ai, and the fixed point sets of this action are exactly 
the critical submanifolds of /' : A4' -^ M. These are of two types. 

The first type is comprised of the manifolds S[, S'^ C Ai' on which /' 
achieves its absolute minimum and maximum. To be precise, the absolute 
minimum Si = f~^{—l) of / : A^ — > M is a copy of SU{2) consisting of 
quadruples [yli,— l,i,j] parametrized by Ai e SU{2). The action 

a*[Ai, -1, i, j] = [Au -I, i, -j] = [i Ai i-\ -1, i, 3] 

corresponds to the map Adj : SU{2) -^ SU{2) sending Ai to iAii~^. 
The fixed point set of this action, which is S[, is the circle consisting of 
quadruples [e*'^, —1, i, j] parametrized by e*'^. Similarly, 5*3 = /~^(1), where 
/ : A^ — s- M achieves its absolute maximum, gives rise to the circle S'^ 
consisting of quadruples [e*^, 1, i, j]. 

Lemma 6.1. The critical submanifolds S[ and S'^ of f are non- degenerate 
in the Morse-Bott sense, and their indices are respectively and 2. 



Proof. The Hessian of / is negative definite on the normal bundle of 53 C M. 
But then its restriction to the normal bundle of 6*3 C M' , which is the 
Hessian of /', is also negative definite. In particular, 5*3 is non-degenerate, 
and its Morse-Bott index equals the codimension of S3 in A4', which is 2. 
The argument for S[ is analogous. D 

Within f^^{—l, 1), the critical points of / : tM — > M coincide with those 
of the moment map arccos /, hence with the fixed points of the circle action 
on /~^(— 1,1). They form the submanifold S2 C A4 which is a 2-torus 
consisting of quadruples \j, i, z, w] parametrized hy z, w £ C such that 
\z\ = \w\ = 1. Note that / = on 6*2. One can easily see that the fixed 
point set ^2 of the involution a* : S2 ^ S2 given by 

o-*[j, h z, w] = [-j, -i, -z, w] = [j, i, -z, w] 

consists of the two circles [j, i, iti, w] with \w\ = 1. 

Lemma 6.2. The critical submanifold S'2 of /' is non- degenerate in the 
Morse-Bott sense, and its index is equal to 1. 

Proof. According to Frankel [1| , the moment map arccos / is a Morse-Bott 
function on /~^(— 1,1), and hence so is the function /. The involution 
M ^ M given by [^1,51,^2,-62] ^^ [^1,-51,^2,-62] changes sign of /. 
Since ^2 is connected, the index of / is half the rank of the normal bundle 
of S'2 C A^, or 2. The involution a* is antisymplectic, see Lemma [3. 3 1 hence 
we can apply [3, Proposition 2.2] to the moment map arccos/. It tells us 
that arccos /' and hence /' are Morse-Bott on S^ , and the index of /' is half 
that of /. D 

Lemma 6.3. The residual involution r : M' -^ M.' acts as the 180° rotation 
on each of the circles S'l and S3, and acts trivially on S2. 

Proof. The circle S3 consists of the quadruples [e*'^, l,i,j] acted upon by r 
as [e*^,l,i,j] —>■ [— e*'^, l,i, j']. The case of SJ is completely similar. The 
manifold S2 consists of the quadruples [j, i, =bi, w] where w is a unit complex 
number. The action of r is given by [j, i, ±i, w] — > [—j, i, ±i, w] = [j, i, ±i, w], 
after conjugating by i. D 



7. The Morse-Bott spectral sequence 

As we have seen, the critical set of the Morse-Bott function /' : A4' — > M 
is a disjoint union S[ U S2 U S'^, where the index of S'p is equal to p — 1, 
and the restriction of /' to each of the S' is constant, f'{S') = p — 2 for 
p = 1, 2, 3. Let Xj = j — 3/2 for j = 0, 1, 2, 3, and consider the filtration 

= ?7^ C U[ C U^ C U'^ = M' 

of M.' by the open sets [/' = (/')~"'^(xo,Xj). The Morse Lemma implies 
that, up to homotopy, [/' is a complex obtained from C^,'„i by attaching, 
along its boundary, the disc bundle associated to the negative normal bundle 
over S'/ whose fibers are the negative definite subspaces of the Hessian of 
/' on S'j. The Ei term of the Morse-Bott spectral sequence associated 
with this filtration is of the form shown in Figure [2l where the {p — l)st 
column represents H*{Sp) for p = 1, 2, 3. This spectral sequence converges 
to H*{M'). 



Figure 2. 

Observe that the filtration is preserved by the residual Z/2 action r : 
A4' -^ M' hence r induces an automorphism r* of the above spectral se- 
quence. Our next task will be to compute the differentials, of which only di 
and d2 are not obviously zero. 



8. Differentials di 

The differential di : E^' — > E^' must vanish because H^{Ai') 7^ 0. In 
particular, we conclude that H^{M') = Z and hence M' is connected. Simi- 
larly, the differential di : E^' —>■ E-^^' must vanish because Ai' is orientable, 
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see Corollary [321 so H'^(M') = Z. In fact, we will show that the remaining 
differentials di also vanish but this will take some effort. 

Let us first consider the differential di : E-^' -^ E^' . It is represented as 
the composition 

H\S',) - H\U^,U[) - H\U^) -. H\U',,U',) - H\S',). (2) 

Here, the second and third arrows are portions of the long exact sequences 
of the respective pairs hence they commute with the automorphism r* . The 
first and the last arrows are Thorn isomorphisms, therefore, their behavior 
with respect to r* is determined by how r acts on 52 and 53 and on their 
negative normal bundles. According to Lemma [6.3l the action of r* on both 
H^iS'^) and H^{S'^) is trivial. 

Lemma 8.1. The involution r acts as minus identity on the normal bundle 
of S'2 C A4' and, in particular, on its negative normal bundle. 

Proof. This will follow from the fact that r : Ai ^ A4 given by the formula 
([T]) is an involution and that its fixed point set coincides with ^2. To show 
the latter, suppose that [^1,51,^2,-62] = [—Ai,Bi,A2,B2]. Then there 
is "u G SU{2) such that uAi = —Aiu and u commutes with Bi, A2 and 
B2. Since {Ai, Bi, A2, B2) is an irreducible representation, we conclude that 
ti^ = 1 or n^ = — 1. The former cannot happen because then —Ai = Ai and 
Ai = 0, a contradiction with Ai € SU{2). Therefore, we have u^ = —1, and 
u = ±i after conjugation. The fact that u = ±i commutes with Bi, A2 and 
B2 means that all three of them are unit complex numbers. In fact, one can 
conjugate by j if necessary to make Bi = e*^ with /3 G [0, vr], perhaps at 
the expense of changing the sign of u. Conjugate further by a unit complex 
number to make Ai = a + bj with real non-negative b, while preserving u, 
Bi, A2 and B2. The relation [Ai,Bi] = [^i,-Bi] • [^2,-82] = -1 then tells 
us that, up to conjugation, Ai = j and Bi = i. Therefore, the only fixed 
points ol r : A4 ^ Ai are of the form [j,i,z,w], with z and w some unit 
complex numbers. This is exactly S2 C Ai. D 

Since the rank of the negative normal bundle to 5*2 is one, r reverses ori- 
entation on the fiber hence r* anticommutes with the first arrow in ([2]). On 
the other hand, S'^ is the absolute maximum hence its negative normal bun- 
dle is the same as its normal bundle. According to Lemma [6.31 the action of 
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r on S'^ is orientation preserving. Since r : A4' -^ A4' preserves orientation 
(which follows for example from Lemma [8. 1|) it must be orientation preserv- 
ing on the negative normal bundle of S'^. Therefore, r* commutes with the 
last arrow in ([2]). In conclusion, di : E^' ^ E^' anticommutes with r* and 
thus must vanish. 

Finally, let us consider the differential di : E^' ^ ii^^' . It is represented 
as the composition 

H\S[) - H\U[,U;,) - H\U[) -. H\U',,U[) - H\S',). (3) 

As before, the two arrows in the middle commute with r*, and the first and 
the last arrows are Thom isomorphisms. The action of r on S'^ is by the 180° 
rotation hence the induced action on H^{S[) is trivial. Since the negative 
normal bundle of S[ is zero dimensional, there is no Thom class to worry 
about and we readily conclude that r* commutes with the first arrow in ^. 
Concerning the last arrow, we already know that r* anticommutes with the 
Thom class of the negative normal bundle of 5*2. On the other hand, r acts 
as the identity on 5*2 and hence as the identity on //^(Sg). This implies that 
r* anticommutes with the last arrow and hence with the composition ([3]). 
Thus di : E^' -^ E^' vanishes. 

9. Differential d2 

The results of the previous section imply that E2 = Ei hence all that 
remains to do is compute the differential d2 '■ E2 — > £"2 ' • ^^ ^^ show that 
the edge homomorphism i* : H^{Ai') — > H^{S'i) in the spectral sequence 
induced by the inclusion i : S[ ^ Ai' is surjective. This will imply that 
(^2 = because a generator of E2 = H^{S[) = Z, must survive in the -Eoo 
term of the spectral sequence, hence it must be in the kernel of d2- Note 
that a similar argument could be used to show vanishing ol di : E^' -^ E^' 
above. 

Remember that S'l consists of the quadruples [e^'^,—l,i,j] € M.'. Con- 
sider the subset 7?.' C ^A' which consists of quadruples [1,51,^2,-62] fixed 
by the involution a* : M' -^ M' . 

Lemma 9.1. TZ' C ^A' is a smoothly embedded 2-sphere which intersects 
S'l C Ai' transversely in exactly one point. 
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Proof. The relation [1,-Bi] • [^2)-S2] = [^2)-S2] = —1 on the points of 
TZ' impUes that, up to conjugation, A2 = i and B2 = j. The fact that 
[l,Bi,i,j] G TZ' is fixed by a* means that a*{[l,Bi,i,j]) = [l,B^^,i,-j] = 
[l,iB^ i^^,i,j] (we used conjugation by i in the last equality). Therefore, 
TZ' is parametrized by Bi € SU{2) such that Bii = i B^ . These are pre- 
cisely the unit quaternions with no i component; they obviously comprise 
an embedded 2-sphere in ^A'. The intersection 7^' n S'^ consists of just one 
point, [1, —l,i,j], and it is obviously transversal. D 

Now, the lemma implies that the natural generator of H^{S'i) = Z is the 
image under i* of the Poincare dual of 7^'. This shows that i* : H^{A4') — > 
H^{S'i) is surjective and thus completes the argument. 

10. Proof of Theorem 11.11 

The Morse-Bott spectral sequence associated with the function /' : A4' -^ 
M converges to H*{M') = H*{J\f'). As we showed in the last two sections, 
all the differentials in this spectral sequence vanish and therefore it collapses 
at the El term, Ei = Eoo- This completes the proof in the case when the 
puncture belongs to the left circle of F' as shown in Figure [H 

If the puncture belongs to the right circle of F', the involution a* : M ^ 
M is given by the formula 

a*[Ai,Bi,A2,B2] = [-BiAiB^\ B^\ B2 A2 B^\ B^\ 

and Theorem 11.11 follows by simply renaming the variables. In the remaining 
case, when p ^ F' belongs to the middle circle, the involution a* : M ^f J\A. 
is given by 

a* [^1,51,^2,^2] = [BiAiB^\ B^\ B2A2B^\ B^']. 

The above proof goes through with minimal changes, which can be safely 
left to the reader. 

Remark 10.1. The regular neighborhood of S'^ L) TZ' = S-^ y S'^ in M' is 
a punctured 5^ x 5^. This implies that A4' splits into a connected sum, 
one of the factors being S^ x 5*^. In particular, the isomorphism H*{J\f') = 
H*{S^ X S^ X S*^) of Theorem 11.11 is not a ring isomorphism. 
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